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ABSTRACT 


In response to the (Scottish) government's decision to 
publish annual educational examination results, on a school-by- school basis, 
a three-year development program was . established by Grampian Education 
Authority (Scotland) and the Centre for Educational Sociology (CES) at the 
University of Edinburgh. In this report, the main findings of the development 
program with respect to two different types of indicators of school 
performance are described. The types of indicators include: young people's 
perceptions of their experiences at school, and the "added value" of schools 
in terms of their pupils' attainment. Key findings include: (1) the majority 

of young people who attended Grampian secondary schools found their schooling 
a worthwhile experience; (2) over three-quarters agreed that "school work was 
worth doing, " "discipline was fair, " and "my teachers helped me to do my 
best"; (3) average attainment in public examinations has risen between 1993 
and 1994; (4) although there are large differences between Grampian schools 

in the average attainment of school leavers, most of the difference between 
schools are attributable to differences in pupil intakes; and (5) league 
tables based solely on published examination results, which do no control for 
pupil intake, give a misleading picture of the relative effectiveness of 
Grampian schools. Tables showing results of the survey are provided. (EH) 
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THE EFFECTIVENESS OF GRAMPIAN 
SECONDARY SCHOOLS 

January 1996 


This report describes the results of a three-year development programme by Grampian 
Education Authority and the Centre for Educational Sociology (CES) at the University of 
Edinburgh. In part, the development programme was a response to the government’s 
decision to publish annual information on public examination results on a school-by- 
school basis. Equally important was the Authority’s wish to base its policies for the 
continuous improvement of its schools on the best possible information, and to ensure 
that the views of parents, teachers and the public were informed by a proper 
understanding of the strengths and limitations of indicators of school quality. 

In this report we describe the main findings of the development programme with respect 
to two different types of indicator of school performance: 

• young people's perceptions of their experiences at school, and 

• the "added value" of schools in terms of their pupils' attainment. 


Some Key Findings 

» The majority of young people who attended Grampian secondary schools found their 
schooling a worthwhile experience. 

» Over three-quarters agreed that "school work was worth doing", "discipline was fair", and "my 
teachers helped me to do my best". 

» Average attainment in public examinations has risen between 1992 and 1994. 

» Although there are large differences between Grampian schools in the average attainment of 
school leavers, most of the difference between schools is attributable to differences in pupil 
intakes. 

» League tables based solely on published examination results, which do not control for pupil 
intake, give a misleading picture of the relative effectiveness of Grampian schools. 


This report was written and produced by Dr Linda Croxford, Centre for Educational Sociology 
and Michael Cowie, Grampian Education Authority. Contributions were also made by Bruce 
Robertson, Grampian Education Authority; Richard Bisset, Kemnay Academy and Michael 
Taylor, Dyce Academy. 
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1. Young People's' Perceptions of their 
Experiences at School 

The survey asked young people a number of 
questions about their experiences of school, and 
asked them to give “an overall view”. Their 
responses are shown in Table 1, in terms of the 
percentages of young people who agreed with 
each statement. Questions from the Grampian 
surveys were subsequently included in the 1993 
Scottish School Leavers Survey; comparative 
figures for the whole of Scotland in 1993 are 
included in Table 1. Grampian school leavers 
are generally more favourable in their 
perceptions of their school experiences than their 
counterparts throughout Scotland. 

More than three-quarters of leavers from 
Grampian schools agreed that “school work was 
worth doing”, “discipline was fair” and “my 
teachers helped me to do my best”. One young 
person explained: 

“I liked the way that it was always easy to 
approach teachers. They always helped 
you as much as possible. If there was 
something you didn ’t understand they were 
willing to explain things if you asked, 
giving up part of their lunch break to do 


Young people wanted their teachers to be 
friendly and approachable, but also to be able to 
maintain an orderly environment. Although 62 
percent of respondents felt that “teachers could 
keep order in class”, some 45 percent considered 
“there were too many troublemakers in my 
class”. 

The majority of school leavers had favourable 
perceptions of their school and most young 
people who attended schools in Grampian would 
echo the words of the youngster who 
commented: 

“When / was at school the teachers really 
encouraged you to do your best and were 
there for you when ever you had a problem 
with school/home etc. They also wanted 
you to do your best and they treated you 
like adults, so I think my school was really 
worthwhile. ” 

It is important to note, however, that there is a 
disaffected minority who feel “my teachers 
didn’t care about me”. Fortunately, there were 
comparatively few young people alienated by 
their experience of school. There are, however, 
a number of issues raised by the survey that 
schools will wish to reflect on and respond to. 


Table 1 : Young people's perceptions of their experiences at school 
(Percentage agreed with statement) 


Grampian Scotland 


School work was worth doing 

82 

80 

Discipline was fair 

79 

74 

My teachers helped me do my best 

75 

71 

Teachers listened to my ideas and views 

72 

66 

If I had a problem there was always a teacher I could talk to 

63 

62 

Teachers could keep order in class 

62 

67 

My school dealt well with any bullying that went on 

59 

51 

My school had a wide choice of after-school activities 

47 

38 

Pupils respected the teachers 

42 

37 

There was vandalism at my school during the school day 

50 

58 

There were too many troublemakers in my class 

45 

44 

Theft among pupils was common at my school 

25 

29 

My teachers didn’t care about me 

16 

16 


Sources: Grampian - Grampian Surveys 1992, 1993, 1994, N (=100%) 1 1 582 
Scotland - Scottish School Leavers Survey 1993, N (=100%) 3469 
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2. Added Value 

Examination results published by the SOED 
Audit Unit can be described as "raw" results, 
because they do not take account of differences 
in pupil intake. In this report we compare the 
“raw” results for Grampian schools with results 
based on the "added value" which does take 
account of pupil intake. 

In this study “added value” is taken to mean the 
contribution made by the school to the progress 
of each pupil, after taking account of differences 
in attainment attributable to: 

• each pupil’s own family background, 

• the school social context, 

• the relative deprivation of the neighbourhood 
in which the school is located, 

• stability and change over three years, 

• gender. 

We estimate added value in terms of the final 
attainment of young people when they leave 
school. Our estimate of added value is different 
from that of the SOED Audit Unit because it 
allows for all the progress made by pupils 
during their time at school, not just between 
Standard Grade and Highers. 


2.1 Measuring added value 

The added value is an estimate. It is found by 
estimating what the SCE attainment of a 
hypothetical “average” pupil would be if: 

i) we sent the same pupil to each school, 

ii) each school had the same social context, 

iii) each school was set in the same 
neighbourhood. 

i) We take account of each pupil's own family 
background 

Research at CES and elsewhere has shown that a 
pupil’s attainment is enhanced or reduced by the 
relative advantage or disadvantage of the 
family’s background circumstances in which 
s/he lives while at school. 

In estimating the added value of schools, we take 
account of family background and estimate the 
attainment of a pupil with “average” family 
background characteristics in each school. By 
this means we can measure the effect if we sent 
the same pupil to each school. 

ii) We take account of the school social context 

Pupils are affected not only by their own family 
background, but also by the family background 
of the other pupils attending their school. 


Family background characteristics which affect pupil attainment 


An advantaged family background A disadvantaged family background 


two natural parents 
below-average number of siblings 
parents who had post-compulsory schooling 
parents who gained advanced qualifications 
parents with non-manual occupations 
father in work 

mother full-time unpaid in the home 
living in a home owned by parents 


lone or step parent 

above-average number of siblings 

parents who left school at the earliest opportunity 

parents without qualifications 

parents with unskilled or manual occupations 

father unemployed 

mother at work 

living in rented accommodation 
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We find that the attainment of all pupils in a 
school is enhanced if the school has many pupils 
from advantaged backgrounds, and conversely 
the attainment of all pupils in a school is 
depressed if the school has few pupils from 
advantaged backgrounds. 

In estimating the added value of schools we 
estimate what the attainment of an “average” 
pupil would be if each school had an “average” 
proportion of pupils from advantaged family 
backgrounds (average social context). By this 
means we can measure the effect if each school 
had the same social context. 

iii) We take account of the effect of the relative 
deprivation of the neighbourhood in which the 
school is located 

Pupils may also be affected by aspects of 
prosperity or deprivation in the neighbourhood 
surrounding their school. We use indicators of 
neighbourhood deprivation derived from the 
1991 census to estimate what the attainment of 
our “average” pupil would be if each school was 
located in an “average” neighbourhood. By this 
means we can measure the effect if each school 
was set in the same neighbourhood. 


A Note of Caution 

Estimates of added value used in this study are 
not perfect measures of school effectiveness. 
The quality of our estimates depends on the 
quality of the measures we use. If we had 
measures of attainment on entry to SI of 
secondary schooling, in addition to measures of 
family background, we could more accurately 
measure the boost given by schools to the 
attainment of each pupil between SI and their 
SCE examinations. However, we know from 
research that measures of pupils’ family 
background are correlated with SI attainment, 
and estimates of added value which control fully 
for school intake do provide good estimates of 
variation between schools. 


2.2 Results , r ; 

Rise in attainment of Grampian school leavers 

The average SCE attainment of Grampian school 
leavers has risen over the three years from 1992 
to 1994. This rise in attainment is the result of 
improved added value in Grampian schools over 
the three years. The majority of schools shared 
in the rise. However, in two schools added value 
did not increase over the three years to the same 
extent as in the rest of Grampian. 

Some of the rise in the added value of schools 
can be attributed to the increased proportion of 
young people who have stayed on to post- 
compulsory schooling, and thereby improved 
their SCE qualifications. School policies for 
improvement may also have contributed to the 
rise. 

Female school leavers have higher attainment 
than males. Part of the difference is caused by 
the greater tendency for girls than boys to stay on 
at school. However, the gender gap narrowed 
slightly over the three years because the 
attainment of males increased more than that of 
females. The rise in attainment of females and 
males of “average” family background is shown 
in Figure 1. 

Figure 1 : Predicted attainment of 'average' 
Grampian school leaver, 1992-1994 
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Differences between schools 

By taking the average results over the three year 
period we obtain stable estimates of the added 
value of schools. Figures 2 and 3 show results 
per school averaged over three years. 
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Figure 2: School differences before taking 
account of intake, 1992-1994 average 



Schools in order of difference 


Figure 2 shows the differences between 
Grampian schools in average examination results 
before taking account of differences in pupil 
intake. (These differences are similar to the 
“raw” examination results used in published 
league tables.) The horizontal line represents the 
Grampian average attainment. Bars on the left 
hand side falling below the horizontal line 
represent schools in which the “raw” 
examination results are below the Grampian 
average. Bars on the right hand side represent 
schools in which “raw” examination results are 
above the Grampian average. We found a 
difference of five and a half Standard Grades in 
the “raw” averages between schools with the 
highest and lowest results, before taking account 
of pupil intake. 


Figure 3: School differences after taking account 
of intake, 1992-1994 average 
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Schools in order of difference 


Figure 3 shows the small difference between 
schools after taking account of pupil intake. It is 
interesting to note that the ordering of schools in 
the two charts does not coincide. There is a 
difference of just one Standard Grade in the 
attainment of an “average” school leaver in the 
school with the highest added value compared to 
the school with the lowest added value (after 
taking account of differences in pupil intakes). 

Statistical tests of significance show that the five 
schools shown on the left of Figure 3 had below 
O erage added value. The three schools shown 

ERIC 


on the right of Figure 3 had above average added 
value. These significant differences between 
schools suggest that there is very notable 
achievement in some schools and scope for 
further improvement in other schools. There is 
no room for complacency. Nevertheless, the 
differences between schools are very small in 
terms of the enhancement of pupil attainment by 
the quality of teaching and learning within each 
school. 

The effect of differences in school social 
context 

There are large differences between Grampian 
schools in the characteristics of the family 
background of pupils attending each school, and 
these cause serious inequalities in pupils’ 
examination results. The attainment of an 
“average pupil” may be raised or lowered by two 
or three Standard Grades by differences in 
school social context. 

The effects of school social context were not the 
same for all pupils. A school with a high social 
context has a greater effect in improving the 
attainment of a pupil whose own family 
background is relatively disadvantaged than of a 
pupil whose own family background is 
advantaged. Conversely, a school with low 
social context has a more depressing effect on 
the attainment of a pupil whose own family 
background is relatively disadvantaged. A pupil 
whose own family background is relatively 
advantaged is less affected by the social context 
of the school. 

In this study we have not examined the process 
by which school social context influences 
attainment. Other research has suggested that in 
schools attended by pupils of advantaged family 
backgrounds pupils have more favourable 
attitudes to academic study and that teachers 
have higher expectations of their pupils. 

The effect of neighbourhood deprivation 

Urban deprivation in the neighbourhood of the 
school has a detrimental effect on pupil 
attainment. This is in addition to the effects of 
school intake characteristics. However, although 
most indicators of local-area deprivation are 
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associated with low pupil attainment, we were 
surprised to find that higher than average local 
unemployment rates are associated with higher 
attainment. It may be that young people in areas 
of high unemployment feel a greater incentive to 
stay on at school and improve their 
qualifications. 

3. Some Implications of this Report 

Grampian Education Authority and its schools 
have broken new ground with this study. The 
study is unique in Scotland and the UK, and has 
been a three-year development programme. The 
analysis has benefited from co-operation and 
dialogue between Grampian Head Teachers, 
Advisers and Officers. This dialogue has led to 
increased awareness of the assumptions and 
limitations of performance indicators. 

• The majority of young people find their 
schooling worthwhile, but a small minority 
are alienated by their school experiences. 

• There are substantial differences among 
Grampian schools in SCE attainment, but 
these differences are very poor indicators 
indeed of the quality of schooling or of the 
improvements a school is making. 

• SCE differences among Grampian schools 
overwhelmingly reflect differences in school 
intake. 

• There are very small differences between 
schools in their added value. In other words, 
there are very small differences between 
schools in the quality of the contribution of 
each school to the attainment of its pupils. 

• Most schools cannot be separated by ranking 
them using added value. 

• Nevertheless, there are significant 
differences between some schools which 
suggests that there is no room for 
complacency. 

One purpose of the study has been to improve 
monitoring of schools by the authority. It 

O 
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therefore provides a base for future monitoring 
by the successor authorities, and has implications 
for their policies. Few Grampian schools differ 
from the regional average in their added value. 
But is the regional average as high as it could 
be? Is there scope for general improvement? An 
analogy might be: are the bottles half full or half 
empty? Consideration should now be given to 
how added value can be raised further. 

A second purpose has been to aid school self- 
evaluation, development planning and 

improvement. So far, the results for each school 
have been confidential to the Head Teacher of 
each school. The results have not been shared 
between schools or within schools. We 

anticipate that now, at the culmination of the 
development programme, Head Teachers will 
wish to share the results and discuss them with 
colleagues. As well as sharing the results, we 
need to share the awareness of the assumptions 
and limitations of the results. 

The findings relating to the perceptions of school 
leavers of their school experiences should give 
schools much encouragement. However, some 
aspects of these results will afford each school 
the opportunity to focus on specific 
improvement initiatives within the school; in 
particular, how can the school guard against the 
alienation felt by a minority of pupils? 

The findings regarding added value have pointed 
to the many factors that contribute to pupil 
attainment. Pupils may suffer disadvantage 
because of their own family background, because 
of the relative disadvantage of their peers and 
also because of neighbourhood deprivation. 
These findings raise questions about the role of 
schools in countering disadvantage. 

We anticipate that schools will use these 
findings to identify potential under-achievement 
among their pupils, and improve their ways of 
helping these pupils to learn. But the problem 
needs the consideration of Grampian and its 
successor authorities. Large differences between 
schools in their social context are a major source 
of inequality within the region. This should 
form the basis for discussion of social equity and 
the nature of disadvantage in Grampian schools 
and in the new Education Authorities. 
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About this study 

There were three annual postal surveys of 
all school leavers from Education Authority 
schools in the region. The surveys in 1992, 
1993 and 1994 related to leavers from 
sessions 1991/2, 1992/3 and 1993/4 

respectively. The surveys described the 
characteristics, experiences, attainment and 
post-school destinations of Grampian 
school leavers. 

Tables showing many of the results of each 
survey have been provided to Head 
Teachers, to enable them to compare the 
responses of leavers from their own school 
to those for the whole of Grampian Region. 
Results for each school are confidential to 
the school. 


Further Reading 

McPherson, A. (1992) "Measuring Added Value 
in Schools", National Commission on 
Education Briefing No 1, (London: National 
Commission on Education). 

Willms, J.D. (1992) Monitoring School 
Performance, (Lewes: Falmer). 


CES Briefings 

This new series will provide regular information 
about the work of the Centre for Educational 
Sociology. CES Briefings are designed to make 
our research accessible to a wider audience, 
and to enhance its value for policy and practice. 
The following Briefings are also available, free 
of charge, from the CES: 

No 1: "Participation of 16-19 year-olds in 
education and training: recent trends" by Paula 
Surridge and David Raffe. 

No 2: "Leaving home" by Gill Jones. 

No. 3: "A Curriculum for All?" by Linda Croxford. 

Other CES Briefings will cover guidance, young 
people's incomes and other areas of our work. 


Further Details 

For further details contact Linda Croxford, 
Centre for Educational Sociology, Tel 0131 650 
4202, E-mail L.Croxford@ed.ac.uk. The views 
expressed are those of the author. 
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CES 

Centre for Educational Sociology 
The University of Edinburgh 
7 Buccleuch Place 
Edinburgh EH8 9LW 
Scotland 
O 



Telephone UK 0131 650 1000 
or direct UK 0131 650 4186/7 
Fax UK 0131 668 3263 
email CES@ed.ac.uk 
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